Cultural traits operating in senders are driving forces of cultural evolution.

We introduce a mathematical model of cultural evolution to study cultural traits that shape how individuals exchange information. Current theory focuses on traits that influence the reception of information (receiver traits), such as evaluating whether information represents the majority or stems from a trusted source. Our model shifts the focus from the receiver to the sender of cultural information and emphasizes the role of sender traits, such as communicability or persuasiveness. Here, we show that sender traits are probably a stronger driving force in cultural evolution than receiver traits. While receiver traits evolve to curb cultural transmission, sender traits can amplify it and fuel the self-organization of systems of mutually supporting cultural traits, including traits that cannot be maintained on their own. Such systems can reach arbitrary complexity, potentially explaining uniquely human practical and mental skills, goals, knowledge and creativity, independent of innate factors. Our model incorporates social and individual learning throughout the lifespan, thus connecting cultural evolutionary theory with developmental psychology. This approach provides fresh insights into the trait-individual duality, that is, how cultural transmission of single traits is influenced by individuals, who are each represented as an acquired system of cultural traits.

Sequence representation as an early step in the evolution of language.

Human language is unique in its compositional, open-ended, and sequential form, and its evolution is often solely explained by advantages of communication. However, it has proven challenging to identify an evolutionary trajectory from a world without language to a world with language, especially while at the same time explaining why such an advantageous phenomenon has not evolved in other animals. Decoding sequential information is necessary for language, making domain-general sequence representation a tentative basic requirement for the evolution of language and other uniquely human phenomena. Here, using formal evolutionary analyses of the utility of sequence representation we show that sequence representation is exceedingly costly and that current memory systems found in animals may prevent abilities necessary for language to emerge. For sequence representation to evolve, flexibility allowing for ignoring irrelevant information is necessary. Furthermore, an abundance of useful sequential information and extensive learning opportunities are required, two conditions that were likely fulfilled early in human evolution. Our results provide a novel, logically plausible trajectory for the evolution of uniquely human cognition and language, and support the hypothesis that human culture is rooted in sequential representational and processing abilities.

How associations become behavior.

The Rescorla and Wagner (1972) model is the first mathematical theory to explain associative learning in the presence of multiple stimuli. Its main theoretical construct is that of associative strength, but this is connected to behavior only loosely. We propose a model in which behavior is described by a collection of Poisson processes, each with a rate proportional to an associative strength. The model predicts that the time between behaviors follows an exponential or hypoexponential distribution. This prediction is supported by two data sets on autoshaped and instrumental behavior in rats.

A test of memory for stimulus sequences in great apes.

Identifying cognitive capacities underlying the human evolutionary transition is challenging, and many hypotheses exist for what makes humans capable of, for example, producing and understanding language, preparing meals, and having culture on a grand scale. Instead of describing processes whereby information is processed, recent studies have suggested that there are key differences between humans and other animals in how information is recognized and remembered. Such constraints may act as a bottleneck for subsequent information processing and behavior, proving important for understanding differences between humans and other animals. We briefly discuss different sequential aspects of cognition and behavior and the importance of distinguishing between simultaneous and sequential input, and conclude that explicit tests on non-human great apes have been lacking. Here, we test the memory for stimulus sequences-hypothesis by carrying out three tests on bonobos and one test on humans. Our results show that bonobos' general working memory decays rapidly and that they fail to learn the difference between the order of two stimuli even after more than 2,000 trials, corroborating earlier findings in other animals. However, as expected, humans solve the same sequence discrimination almost immediately. The explicit test on whether bonobos represent stimulus sequences as an unstructured collection of memory traces was not informative as no differences were found between responses to the different probe tests. However, overall, this first empirical study of sequence discrimination on non-human great apes supports the idea that non-human animals, including the closest relatives to humans, lack a memory for stimulus sequences. This may be an ability that sets humans apart from other animals and could be one reason behind the origin of human culture.

Models of conditioned reinforcement and abnormal behaviour in captive animals.

Abnormal behaviours are common in captive animals, and despite a lot of research, the development, maintenance and alleviation of these behaviours are not fully understood. Here, we suggest that conditioned reinforcement can induce sequential dependencies in behaviour that are difficult to infer from direct observation. We develop this hypothesis using recent models of associative learning that include conditioned reinforcement and inborn facets of behaviour, such as predisposed responses and motivational systems. We explore three scenarios in which abnormal behaviour emerges from a combination of associative learning and a mismatch between the captive environment and inborn predispositions. The first model considers how abnormal behaviours, such as locomotor stereotypies, may arise from certain spatial locations acquiring conditioned reinforcement value. The second model shows that conditioned reinforcement can give rise to abnormal behaviour in response to stimuli that regularly precede food or other reinforcers. The third model shows that abnormal behaviour can result from motivational systems being adapted to natural environments that have different temporal structures than the captive environment. We conclude that models including conditioned reinforcement offer an important theoretical insight regarding the complex relationships between captive environments, inborn predispositions, and learning. In the future, this general framework could allow us to further understand and possibly alleviate abnormal behaviours.

Modeling relational responding with artificial neural networks.

Relational responding refers to behavior that conforms to a rule for com- paring stimuli. Lazareva et al. (2014) trained pigeons to choose either the smaller or the larger of two circles, using 1-3 pairs of circles for training and 17-19 new pairs for testing. The pigeons showed relational responding on some test pairs and systematic failures on others. We present a simple artificial neural network model that reproduces the animals' behavior well, similarly to Lazareva et al.'s (2014) statistical model based on stimulus features and stimulus relationships. We analyze how the network model gener- alizes from training to test stimuli, and show that it can reconcile contrasting ideas about relational responding from the seminal works by Köhler (1929, 1918/1938, 1924), positing that animals can learn relational rules such as "choose the larger stimulus," and Spence (1937), positing that relational re- sponding can be explained based on stimulus generalization.

A response function that maps associative strengths to probabilities.

Bridging associative and normative theories of animal learning, I show that an associative system can behave as if performing probabilistic inference by using the function f(V) = 1 - e-cV to transform associative strengths (V) into response probabilities. For example, using this function, an associative system can respond normatively to a compound stimulus AB, given previous separate experiences with the components A and B. The CR probability formulae that result from the proposed function have a normative interpretation in terms of statistical decision theory. The formulae also suggest a normative interpretation of stimulus generalization as a heuristic to infer whether different stimuli are likely to convey redundant or independent information about reinforcement. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Pavlovian summation: Data and theory.

In summation experiments, responding to a compound stimulus is assessed after conditioning a response to each of its components. This simple experiment poses significant challenges to models of associative learning because of substantial variability in results. Here, I introduce a new method to quantify generalization from components to compound in summation experiments, which I apply to over 250 measurements of summation in rabbits, pigeons, rats, and humans. The analysis confirms that more summation occurs with stimuli from different rather than from the same sensory modality, although this is not the sole determinant of summation. A theoretical analysis shows that this finding is best accounted for by a model that includes both element sharing (Rescorla & Wagner, 1972) and element replacement (Brandon et al., 2000) in stimulus representations. I point out remaining gaps in our empirical and theoretical understanding of summation. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A-learning: A new formulation of associative learning theory.

We present a new mathematical formulation of associative learning focused on non-human animals, which we call A-learning. Building on current animal learning theory and machine learning, A-learning is composed of two learning equations, one for stimulus-response values and one for stimulus values (conditioned reinforcement). A third equation implements decision-making by mapping stimulus-response values to response probabilities. We show that A-learning can reproduce the main features of: instrumental acquisition, including the effects of signaled and unsignaled non-contingent reinforcement; Pavlovian acquisition, including higher-order conditioning, omission training, autoshaping, and differences in form between conditioned and unconditioned responses; acquisition of avoidance responses; acquisition and extinction of instrumental chains and Pavlovian higher-order conditioning; Pavlovian-to-instrumental transfer; Pavlovian and instrumental outcome revaluation effects, including insight into why these effects vary greatly with training procedures and with the proximity of a response to the reinforcer. We discuss the differences between current theory and A-learning, such as its lack of stimulus-stimulus and response-stimulus associations, and compare A-learning with other temporal-difference models from machine learning, such as Q-learning, SARSA, and the actor-critic model. We conclude that A-learning may offer a more convenient view of associative learning than current mathematical models, and point out areas that need further development.

Social learning through associative processes: a computational theory.

Social transmission of information is a key phenomenon in the evolution of behaviour and in the establishment of traditions and culture. The diversity of social learning phenomena has engendered a diverse terminology and numerous ideas about underlying learning mechanisms, at the same time that some researchers have called for a unitary analysis of social learning in terms of associative processes. Leveraging previous attempts and a recent computational formulation of associative learning, we analyse the following learning scenarios in some generality: learning responses to social stimuli, including learning to imitate; learning responses to non-social stimuli; learning sequences of actions; learning to avoid danger. We conceptualize social learning as situations in which stimuli that arise from other individuals have an important role in learning. This role is supported by genetic predispositions that either cause responses to social stimuli or enable social stimuli to reinforce specific responses. Simulations were performed using a new learning simulator program. The simulator is publicly available and can be used for further theoretical investigations and to guide empirical research of learning and behaviour. Our explorations show that, when guided by genetic predispositions, associative processes can give rise to a wide variety of social learning phenomena, such as stimulus and local enhancement, contextual imitation and simple production imitation, observational conditioning, and social and response facilitation. In addition, we clarify how associative mechanisms can result in transfer of information and behaviour from experienced to naive individuals.

On the role of responses in Pavlovian acquisition.

A defining feature of Pavlovian conditioning is that the unconditioned stimulus (US) is delivered whether or not the animal performs a conditioned response (CR). This has lead to the question: Does CR performance play any role in conditioning? Between the 1930s and 1970s, a consensus emerged that CR acquisition is driven by CS-US (CS, conditioned stimulus) experiences, and that CRs play a minimal role, if any. Here we revisit the question and present 2 new quantitative methods to evaluate whether CRs influence the course of learning. Our results suggest that CRs play an important role in Pavlovian acquisition, in such paradigms as rabbit eye blink conditioning, pigeon autoshaped key pecking, and rat autoshaped lever pressing and magazine entry. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

A Comparison of Individual Learning and Social Learning in Zebrafish Through an Ethorobotics Approach.

Social learning is ubiquitous across the animal kingdom, where animals learn from group members about predators, foraging strategies, and so on. Despite its prevalence and adaptive benefits, our understanding of social learning is far from complete. Here, we study observational learning in zebrafish, a popular animal model in neuroscience. Toward fine control of experimental variables and high consistency across trials, we developed a novel robotics-based experimental test paradigm, in which a robotic replica demonstrated to live subjects the correct door to join a group of conspecifics. We performed two experimental conditions. In the individual training condition, subjects learned the correct door without the replica. In the social training condition, subjects observed the replica approaching both the incorrect door, to no effect, and the correct door, which would open after spending enough time close to it. During these observations, subjects could not actively follow the replica. Zebrafish increased their preference for the correct door over the course of 20 training sessions, but we failed to identify evidence of social learning, whereby we did not register significant differences in performance between the individual and social training conditions. These results suggest that zebrafish may not be able to learn a route by observation, although more research comparing robots to live demonstrators is needed to substantiate this claim.

ecco: An error correcting comparator theory.

Building on the work of Ralph Miller and coworkers (Miller and Matzel, 1988; Denniston et al., 2001; Stout and Miller, 2007), I propose a new formalization of the comparator hypothesis that seeks to overcome some shortcomings of existing formalizations. The new model, dubbed ecco for "Error-Correcting COmparisons," retains the comparator process and the learning of CS-CS associations based on contingency. ecco assumes, however, that learning of CS-US associations is driven by total error correction, as first introduced by Rescorla and Wagner (1972). I explore ecco's behavior in acquisition, compound conditioning, blocking, backward blocking, and unovershadowing. In these paradigms, ecco appears capable of avoiding the problems of current comparator models, such as the inability to solve some discriminations and some paradoxical effects of stimulus salience. At the same time, ecco exhibits the retrospective revaluation phenomena that are characteristic of comparator theory.

Can squirrel monkeys learn an ABnA grammar? A re-evaluation of Ravignani et al. (2013).

Ravignani et al. (2013) habituated squirrel monkeys to sound sequences conforming to an ABnA grammar (n = 1, 2, 3), then tested them for their reactions to novel grammatical and non-grammatical sequences. Although they conclude that the monkeys "consistently recognized and generalized the sequence ABnA," I remark that this conclusion is not robust. The statistical significance of results depends on specific choices of data analysis, namely dichotomization of the response variable and omission of specific data points. Additionally, there is little evidence of generalization to novel patterns (n = 4, 5), which is important to conclude that the monkeys recognized the ABnA grammar beyond the habituation patterns. Lastly, many test sequences were perceptually similar to habituation sequences, raising the possibility that the monkeys may have generalized based on perceptual similarity rather than based on grammaticality.

Memory for stimulus sequences: a divide between humans and other animals?

Humans stand out among animals for their unique capacities in domains such as language, culture and imitation, yet it has been difficult to identify cognitive elements that are specifically human. Most research has focused on how information is processed after it is acquired, e.g. in problem solving or 'insight' tasks, but we may also look for species differences in the initial acquisition and coding of information. Here, we show that non-human species have only a limited capacity to discriminate ordered sequences of stimuli. Collating data from 108 experiments on stimulus sequence discrimination (1540 data points from 14 bird and mammal species), we demonstrate pervasive and systematic errors, such as confusing a red-green sequence of lights with green-red and green-green sequences. These errors can persist after thousands of learning trials in tasks that humans learn to near perfection within tens of trials. To elucidate the causes of such poor performance, we formulate and test a mathematical model of non-human sequence discrimination, assuming that animals represent sequences as unstructured collections of memory traces. This representation carries only approximate information about stimulus duration, recency, order and frequency, yet our model predicts non-human performance with a 5.9% mean absolute error across 68 datasets. Because human-level cognition requires more accurate encoding of sequential information than afforded by memory traces, we conclude that improved coding of sequential information is a key cognitive element that may set humans apart from other animals.

The power of associative learning and the ontogeny of optimal behaviour.

Behaving efficiently (optimally or near-optimally) is central to animals' adaptation to their environment. Much evolutionary biology assumes, implicitly or explicitly, that optimal behavioural strategies are genetically inherited, yet the behaviour of many animals depends crucially on learning. The question of how learning contributes to optimal behaviour is largely open. Here we propose an associative learning model that can learn optimal behaviour in a wide variety of ecologically relevant circumstances. The model learns through chaining, a term introduced by Skinner to indicate learning of behaviour sequences by linking together shorter sequences or single behaviours. Our model formalizes the concept of conditioned reinforcement (the learning process that underlies chaining) and is closely related to optimization algorithms from machine learning. Our analysis dispels the common belief that associative learning is too limited to produce 'intelligent' behaviour such as tool use, social learning, self-control or expectations of the future. Furthermore, the model readily accounts for both instinctual and learned aspects of behaviour, clarifying how genetic evolution and individual learning complement each other, and bridging a long-standing divide between ethology and psychology. We conclude that associative learning, supported by genetic predispositions and including the oft-neglected phenomenon of conditioned reinforcement, may suffice to explain the ontogeny of optimal behaviour in most, if not all, non-human animals. Our results establish associative learning as a more powerful optimizing mechanism than acknowledged by current opinion.

Modeling the genealogy of a cultural trait.

The mathematical study of genealogies has yielded important insights in population biology, such as the ability to estimate the time to the most recent common ancestor (MRCA) of a sample of genetic sequences or of a group of individuals. Here we introduce a model of cultural genealogies that is a step toward answering similar questions for cultural traits. In our model individuals can inherit from a variable, potentially large number of ancestors, rather than from a fixed, small number of ancestors (one or two) as is typical of genetic evolution. We first show that, given a sample of individuals, a cultural common ancestor does not necessarily exist. We then introduce a related concept: the most recent unique ancestor (MRUA), i.e., the most recent single individual who is the earliest cultural ancestor of the sample. We show that, under neutral evolution, the time to the MRUA can be staggeringly larger than the time to MRCA in a single ancestor model, except when the average number of learning opportunities per individuals is small. Our results point out that the properties of cultural genealogies may be very different from those of genetic genealogies, with potential implications for reconstructing the histories of cultural traits.

Solution of the comparator theory of associative learning.

We derive an analytical solution of the comparator theory of associative learning, as formalized by Stout and Miller (2007). The solution enables us to calculate exactly the predicted responding to stimuli in any experimental design and for any choice of model parameters. We illustrate its utility by calculating the predictions of comparator theory in some paradigmatic designs: acquisition of conditioned responses, compound conditioning, blocking, unovershadowing, and backward blocking. We consider several versions of the theory: first-order comparator theory (close to the original ideas of Miller & Matzel, 1988), second-order comparator theory (Denniston, Savastano, & Miller, 2001), and sometimes-competing retrieval (Stout & Miller, 2007). We show that all versions of comparator theory make a number of surprising predictions, some of which appear hard to reconcile with empirical data. Our solution paves the way for a fuller understanding of the theory and for its empirical evaluation.

Animal memory: A review of delayed matching-to-sample data.

We performed a meta-analysis of over 90 data sets from delayed matching-to-sample (DMTS) studies with 25 species (birds, mammals, and bees). In DMTS, a sample stimulus is first presented and then removed. After a delay, two (or more) comparison stimuli are presented, and the subject is rewarded for choosing the one matching the sample. We used data on performance vs. delay length to estimate two parameters informative of working memory abilities: the maximum performance possible with no delay (comparison stimuli presented as soon as the sample is removed), and the rate of performance decay as the delay is lengthened (related to memory span). We conclude that there is little evidence that zero-delay performance varies between these species. There is evidence that pigeons do not perform as well as mammals at longer delay intervals. Pigeons, however, are the only extensively studied bird, and we cannot exclude that other birds may be able to bridge as long a delay as mammals. Extensive training may improve memory, although the data are open to other interpretations. Overall, DMTS studies suggest memory spans ranging from a few seconds to several minutes. We suggest that observations of animals exhibiting much longer memory spans (days to months) can be explained in terms of specialized memory systems that deal with specific, biologically significant information, such as food caches. Events that do not trigger these systems, on the other hand, appear to be remembered for only a short time. This article is part of a Special Issue entitled: In Honor of Jerry Hogan.

Dog movie stars and dog breed popularity: a case study in media influence on choice.

Fashions and fads are important phenomena that influence many individual choices. They are ubiquitous in human societies, and have recently been used as a source of data to test models of cultural dynamics. Although a few statistical regularities have been observed in fashion cycles, their empirical characterization is still incomplete. Here we consider the impact of mass media on popular culture, showing that the release of movies featuring dogs is often associated with an increase in the popularity of featured breeds, for up to 10 years after movie release. We also find that a movie's impact on breed popularity correlates with the estimated number of viewers during the movie's opening weekend--a proxy of the movie's reach among the general public. Movies' influence on breed popularity was strongest in the early 20th century, and has declined since. We reach these conclusions through a new, widely applicable method to measure the cultural impact of events, capable of disentangling the event's effect from ongoing cultural trends.